Emotion-Aware Speaker Identification with Transfer Learning

Speech is a natural communication method used by humans. Speaker identification (SI) technology based on human speech has been as an entry point for many human-computer-interaction applications. The performance of SI models can degrade when dealing with expressive uttered in emotional situations because emotion databases do not have sufficient data to train various states. Generally, are trained using relatively more samples “neutral” than other classes. In this study, we propose emotion-aware (em-SI) that uses emotion-embedding vector generated from pre-trained recognition (SER) model along the acoustic features data. We assess individual English and Korean corpora confirm proposed provides improved multilingual corpora. evaluation results show accuracy em-SI Emotion Multimodal Database (KEMDy19) 3.2%, average speaker verification (SV) terms equal error rate (EER) was 1.3% compared baseline model. visualization embedding shows maps space where both information simultaneously represented. Through experiments conducted confirmed model, which learns integrating information, speech.